Supervised Naive Bayes Parameters

Bayesian network models are widely used for supervised prediction tasks such as classification. The Naive Bayes (NB) classifier in particular has been successfully applied in many fields. Usually its parameters are determined using ‘unsupervised’ methods such as likelihood maximization. This can lead to seriously biased prediction, since the independence assumptions made by the NB model rarely ever hold. It has not been clear though, how to find parameters maximizing the supervised likelihood or posterior globally. In this paper we show, how this supervised learning problem can be solved efficiently. We introduce an alternative parametrization in which the supervised likelihood becomes concave. From this result it follows that there can be at most one maximum, easily found by local optimization methods. We present test results that show this is feasible and highly beneficial.